As consumer collections of media assets, such as still images or videos, continue to grow, access and retrieval becomes increasingly daunting. The problem is compounded by the ease at which digital content may be captured and stored, enabling people to capture far more content than they would have with prior, film-based means of capture. Such content accumulates in the electronic equivalent of a picture “shoebox”—unused and unlooked at for years, due to the difficulty of retrieving content from specific events from such, generally unorganized, collections. Digital imaging is still a relatively new technology and most individual collections cover less than a decade; the problem will only worsen as digital imaging is used to record a lifetime of memories.
Such content may be manually annotated with text labels and stored in a database to be retrieved by keyword. However, manual annotation is a tedious task seldom performed by most consumers. With current interfaces, most people cannot be expected to invest a large amount of upfront effort to annotate their images in the hope of facilitating future retrieval. Research continues in algorithms to automatically extract semantic information from assets, including scene classifiers, activity recognizers and people recognizers. A high degree of accuracy in such algorithms remains elusive, particularly for algorithms attempting to extract higher-level conceptual information. The best source for such conceptual semantic information remains the users themselves; the challenge is to obtain such information in an unobtrusive manner and in a way that minimizes the amount of effort required by the user.
Earlier work described in U.S. Pat. No. 7,028,253 B1 to Lieberman et al. attempted to do just that—to obtain semantic information from the user by providing the user with a tool for automatic annotation and retrieval. While the '253 patent provides beneficial solutions, a continuing need in the art exists for improved solutions to retrieving and annotating media assets.